Scholarly Tweets: Measuring Research Impact via Altmetrics

Last spring a faculty member asked me to find their h-index, collect the Impact Factor (IF) for all the journals in which they were published, and compile the number of citations each of their articles had received. When I presented the results, they insisted their work was more influential than what the metrics suggested. I explained why IFs for specialized journals were lower than more generalized publications such as Science or Nature, and why citation counts for recent articles need time to grow. I introduced the topic of altmetrics, or alternative metrics, and how they could help this faculty member build a more complete picture of their research impact. Altmetrics measure research impact by including references outside of traditional scholarly publishing [1]. These social web metrics were first proposed in 2010 as a response to scholars moving their work online [2]. Altmetrics analyze tweets, blogs, presentations, news articles, comments, or any social commentary about a diverse group of scholarly activities that are captured on the web [2]. As librarians have been explaining for years, there are limits to what IF and h-index figures can tell us. A junior faculty member may have created and shared hundreds of captivating lectures online but only published a few articles. That teaching is not reflected in their h-index. They may author a widely followed blog in which they engage with an audience of academic peers, but there is no IF for the blog. As numbers of Twitter followers or Facebook friends quantify social media activity, altmetrics measure and rank researcher output, impact, and influence from the social web. Before continuing, I should declare any possible conflicts of interest. I have been a tester for Altmetric.com since September 2013, and I have enjoyed free access to the Altmetric.com Explorer. Aside from that association, I am a health librarian working in an academic research centre without any financial interest in altmetric products. I am writing from the perspective of an information specialist who uses altmetrics to track research publications, to assess the impact of my centre’s social media communication strategy, and to plan future knowledge dissemination projects. I first heard about altmetrics at the Medical Library Association conference in 2013. The topic of this intriguing new cross-section of social media and article metrics was heard at numerous sessions. Altmetric start-ups were pushing products that measured research in a different way, and librarians were excited to learn more. The obvious caveat about altmetrics is that they are only valid and valuable for the most recent publications [3, 4]. A 2013 publication in Annals of Internal Medicine on lifestyle interventions for diabetes [5] received a score of 148 by the Altmetric.com Explorer. The score was calculated based on 128 Twitter users sharing the publication, two bloggers citing the article, and seven Facebook users mentioning it. In context, the article was amongst the highest ever scored in Annals of Internal Medicine, (ranked 37 out of 2470). In contrast, a 2005 publication from the same journal on coronary artery disease [6] received no score. Social media mentions are rare for articles published prior to 2011 [4, 7], and altmetric products often exclude older datasets in their analysis [8]. Some altmetric tools, such as Altmetric.com in the aforementioned example, tell us about individual articles and others tell us about researchers. Researcher-focused products, such as ImpactStory and ResearchGate, resemble familiar social networking sites in that they rely on contributors creating and maintaining personal profiles. The evolution from Facebook to LinkedIn to ImpactStory makes logical sense. User-contributed profiles became online résumés and then dynamic curricula vitae with embedded metrics for research products. For researcherfocused altmetric tools, older publications, presentations, and products can be manually added. These products that tell us about researchers are more likely to include contributions prior to 2011, and for that reason they are better for analyzing research output over time than articlefocused altmetric tools. Because different altmetric tools tell different stories, the landscape is full of start-ups positioning themselves as